Space Dilation in Polynomial-Time Perceptron Algorithms

This talk surveys applications of space dilation in the perceptron-like algorithms for solving systems of linear inequalities in polynomial time: from the ellipsoid algorithm to the modified perceptron algorithm and the perceptron rescaling algorithm. It also suggests a more general version of the latter algorithm and poses an open problem about the lowest complexity of this approach. Space dilation is a linear non-orthogonal space transformation for subgradient descent algorithms to handle the case when the subgradient is almost orthogonal to the direction towards the optimum [5]. It aims to reduce the angle between the anti-subgradient and that direction by applying a linear operator that changes the metric of the space. It was exploited in the ellipsoid algorithm to make its running time polynomial [3]. We consider a problem of solving a system of linear inequalities by the perceptron algorithm (a simple incremental subgradient descent method) [4], which may have an exponential complexity in the worst case. The rescaling procedure, which is a special case of the space dilation, is designed for the perceptron rescaling algorithm [2]. It uses the modified perceptron algorithm [1] to generate the direction of rescaling and converges in polynomial time with high probability. We suggest a more general formulation of this approach, which views certain assumptions in the algorithm as parameters, show how its convergence and complexity depends on these parameters, and pose an open problem about their optimal values.